<html>
<head><meta charset="utf-8"><title>Abomonation UB · t-lang/wg-unsafe-code-guidelines · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/index.html">t-lang/wg-unsafe-code-guidelines</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation.20UB.html">Abomonation UB</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="175068684"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation%20UB/near/175068684" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Frank McSherry <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation.20UB.html#175068684">(Sep 06 2019 at 14:16)</a>:</h4>
<p>Recently brought to my attention by <span class="user-mention" data-user-id="211871">@Hadrien Grasland</span> in (<a href="https://github.com/TimelyDataflow/abomonation/issues/17" target="_blank" title="https://github.com/TimelyDataflow/abomonation/issues/17">https://github.com/TimelyDataflow/abomonation/issues/17</a>) that reconstructing a <code>&amp; T</code> is 1. UB in abomonation because of (at least) the <code>dereferenceable</code> tag promised as soon as a <code>&amp;mut  T</code> is instantiated (its fields are then correctly populated, but are initially invalid), and 2. hard to do with <code>NonNull&lt;T&gt;</code> as navigating the structure of <code>T</code> is challenging without a reference due to (I imagine) the lack of field dereferences to <code>NonNull&lt;FieldOfT&gt; </code>.</p>
<p>No clue if this is on anyone's radar, but I personally hold abomonation as a bit of a test for whether a language is suitable for a certain type of systems programming. If it is hard to go from bytes to typed data, without several allocations along the way, that is worth understanding as a something that is possibly being traded away. No intent to claim that either is the right trade-off, just calling out that there is one here either to make, or work to avoid making!</p>



<a name="175071838"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation%20UB/near/175071838" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Hanna Kruppe <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation.20UB.html#175071838">(Sep 06 2019 at 14:50)</a>:</h4>
<p>FWIW problem 2 will be solved (though not necessarily very ergonomically for <code>NonNull</code>, raw pointers have it better) by &lt;<a href="https://github.com/rust-lang/rfcs/pull/2582" target="_blank" title="https://github.com/rust-lang/rfcs/pull/2582">https://github.com/rust-lang/rfcs/pull/2582</a>&gt; which enabled field projections through raw pointers without involving any references or other UB traps</p>



<a name="175085003"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation%20UB/near/175085003" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Shnatsel <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation.20UB.html#175085003">(Sep 06 2019 at 17:24)</a>:</h4>
<p><del>FWIW there are also <a href="https://crates.io/crates/plain" target="_blank" title="https://crates.io/crates/plain">https://crates.io/crates/plain</a> and <a href="https://crates.io/crates/zerocopy" target="_blank" title="https://crates.io/crates/zerocopy">https://crates.io/crates/zerocopy</a> crates to do type punning safely. I wonder if the same issue applies to them as well?</del><br>
<del>The latter comes with custom derives making sure it's OK, although author of this PR managed to get the same thing validated using pure <code>macro_rules!</code> in <a href="https://github.com/m4b/goblin/pull/182" target="_blank" title="https://github.com/m4b/goblin/pull/182">https://github.com/m4b/goblin/pull/182</a> </del></p>



<a name="175085197"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation%20UB/near/175085197" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Shnatsel <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation.20UB.html#175085197">(Sep 06 2019 at 17:26)</a>:</h4>
<p>Nevermind, I misunderstood the problem.</p>



<a name="175145214"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation%20UB/near/175145214" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Frank McSherry <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation.20UB.html#175145214">(Sep 07 2019 at 15:42)</a>:</h4>
<blockquote>
<p>FWIW problem 2 will be solved</p>
</blockquote>
<p>This sounds promising, thanks for the heads up! I apologize for the random drive-bys, but it feels like a full-time job to keep up with things here. I'm hoping that dropping in with an anxiously phrased lament isn't taken the wrong way (actual intent: "important to some! if not all!").</p>



<a name="175748193"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation%20UB/near/175748193" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> RalfJ <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation.20UB.html#175748193">(Sep 15 2019 at 13:06)</a>:</h4>
<p><span class="user-mention" data-user-id="116609">@Frank McSherry</span> thanks for the comment, it is much appreciated!</p>



<a name="175748199"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation%20UB/near/175748199" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> RalfJ <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation.20UB.html#175748199">(Sep 15 2019 at 13:06)</a>:</h4>
<p>indeed abomonation is something I'd like Rust to support</p>



<a name="175748204"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation%20UB/near/175748204" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> RalfJ <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation.20UB.html#175748204">(Sep 15 2019 at 13:06)</a>:</h4>
<p>so, when you see things that worry you in that regard, please keep us informed :)</p>



<a name="175748224"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation%20UB/near/175748224" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> RalfJ <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation.20UB.html#175748224">(Sep 15 2019 at 13:07)</a>:</h4>
<p>note that <code>dereferencable</code> just says that the reference points to allocated memory</p>



<a name="175748226"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation%20UB/near/175748226" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> RalfJ <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation.20UB.html#175748226">(Sep 15 2019 at 13:07)</a>:</h4>
<p>but for Rust right now we also demand that it points to something valid at the given type -- that's <a href="https://github.com/rust-lang/unsafe-code-guidelines/issues/77" target="_blank" title="https://github.com/rust-lang/unsafe-code-guidelines/issues/77">https://github.com/rust-lang/unsafe-code-guidelines/issues/77</a></p>



<a name="175748230"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation%20UB/near/175748230" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> RalfJ <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation.20UB.html#175748230">(Sep 15 2019 at 13:07)</a>:</h4>
<p>I suppose that's what you are worried about here?</p>



<a name="175751462"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation%20UB/near/175751462" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Frank McSherry <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation.20UB.html#175751462">(Sep 15 2019 at 14:29)</a>:</h4>
<p>Thanks <span class="user-mention" data-user-id="120791">@RalfJ</span>!</p>
<p>In Abomonation, it's not necessarily the case that the pointer points at allocated memory. We've just pulled a <code>&amp;mut [u8]</code> off the wire, let's say, and would like to start looking at it as a <code>Vec&lt;String&gt;</code>. If we transmute to a <code>&amp;mut Vec&lt;String&gt;</code> with the plan of filling in the pointers, we have a problem as there is some amount of time between the cast and the assignment to the fields. They are never read before being written, but .. I suspect that doesn't save any bacon.</p>



<a name="175751546"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation%20UB/near/175751546" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Frank McSherry <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation.20UB.html#175751546">(Sep 15 2019 at 14:31)</a>:</h4>
<p>More specifically, none of the <em>pointers</em> are read before being written; length fields and such may well be (we could possibly write the pointer first, but we want to read the len to make sure that the extent of memory described is in bounds before forming a vector whose length could overrun our allocation).</p>



<a name="175751600"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation%20UB/near/175751600" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Frank McSherry <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation.20UB.html#175751600">(Sep 15 2019 at 14:32)</a>:</h4>
<p>This is all negotiable if it turns out there is a safer way to go from partially allocated memory to in-memory representations (I'm sure there are rules, that they are subtle, and that in my ignorance I am violating many of them).</p>



<a name="175751772"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation%20UB/near/175751772" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Frank McSherry <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation.20UB.html#175751772">(Sep 15 2019 at 14:37)</a>:</h4>
<p>And, in case it isn't clear, the only reason to have a cast to a mutable reference (as opposed to an assignment from a newly constructed value) is that it is the only way I know to get access to the fields (as I can't guess the layout and sniff around for the length field). Though, another reason might be that with large types only changing the locations that matter could be a win vs overwriting the whole struct.</p>



<a name="175755874"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation%20UB/near/175755874" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> gnzlbg <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation.20UB.html#175755874">(Sep 15 2019 at 16:28)</a>:</h4>
<p>How are you transmuting a <code>&amp;mut [u8]</code> to a <code>&amp;mut Vec&lt;String&gt;</code> ?</p>



<a name="175798538"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation%20UB/near/175798538" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> RalfJ <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation.20UB.html#175798538">(Sep 16 2019 at 09:59)</a>:</h4>
<blockquote>
<p>If we transmute to a &amp;mut Vec&lt;String&gt; with the plan of filling in the pointers</p>
</blockquote>
<p>if you want to fill it, that means this reference <em>does</em> point to 24 bytes worth of allocated memory, right?<br>
"dereferencable" is not recursive. <code>&amp;mut Vec&lt;String&gt;</code> just means "there's 24 bytes worth of memory there".</p>
<p>Well, at least that's the LLVM side of things. Rust has more aggressive UB, but the pointer in a <code>Vec</code> is a raw one, so Rust will not assume that pointer to be valid, and I don't think it ever could.</p>



<a name="175798688"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation%20UB/near/175798688" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> RalfJ <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation.20UB.html#175798688">(Sep 16 2019 at 10:01)</a>:</h4>
<p>so I don't think I see where there are dangling pointers (not pointing to allocated memory) here in a poblematic way. I am more worried about references you are creating <em>into</em> the backing buffer. or things like <code>NonNull</code> being violated (the ptr in a <code>Vec</code> must never be NULL, and that also means that a <code>&amp;mut Vec&lt;String&gt;</code> must never point to data where that ptr is null -- Rust's validity reuqirement <em>is</em> recursive right now)</p>



<a name="175798959"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation%20UB/near/175798959" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> gnzlbg <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation.20UB.html#175798959">(Sep 16 2019 at 10:05)</a>:</h4>
<blockquote>
<p>but the pointer in a Vec is a raw one, so Rust will not assume that pointer to be valid, and I don't think it ever could.</p>
</blockquote>
<p>The pointer in a <code>Vec</code> is a <code>Unique&lt;T&gt;</code>, so Rust will assume a couple of things about it (IIRC <code>Unique</code> is a <code>NonNull&lt;T&gt;</code> wrapper with PhantomData and Send/Sync, so this will impact dropck and other things, but NonNull is probably the most important thing as you mentioned)</p>



<a name="175799298"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation%20UB/near/175799298" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> RalfJ <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation.20UB.html#175799298">(Sep 16 2019 at 10:11)</a>:</h4>
<p>right, I meant that we do not assume the ptr to point to allocated memory</p>



<a name="175799300"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation%20UB/near/175799300" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> RalfJ <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation.20UB.html#175799300">(Sep 16 2019 at 10:11)</a>:</h4>
<p>it actually frequently does not, after <code>Vec::new()</code></p>



<a name="176001898"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation%20UB/near/176001898" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Frank McSherry <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation.20UB.html#176001898">(Sep 18 2019 at 12:57)</a>:</h4>
<blockquote>
<p>if you want to fill it, that means this reference does point to 24 bytes worth of allocated memory, right?</p>
</blockquote>
<p>Oh, yes. It does. I assumed transitivity of <code>dereferenceable</code> (at least, that seemed to be the scary thing raised in the issue), but this is good to know. At least the language of "points to a valid instance" shakes one up on this (probably healthy). Perhaps we are over-anxious about the potential UB here.</p>
<blockquote>
<p>the ptr in a Vec must never be NULL, and that also means that a &amp;mut Vec&lt;String&gt; must never point to data where that ptr is null -- Rust's validity reuqirement is recursive right now</p>
</blockquote>
<p>This is probably not violated, I think. It came up before, but in order to correctly de-abomonate <code>Option&lt;T&gt; </code> types we need to ensure this also. Because it is a memcpy of the source address space, as long as the definition of non-NULL is the same across the encoding and decoding binaries (which, if not then there are other problems) I think the pointers will be non-NULL to potentially invalid memory.</p>



<a name="176002748"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation%20UB/near/176002748" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Frank McSherry <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation.20UB.html#176002748">(Sep 18 2019 at 13:06)</a>:</h4>
<blockquote>
<p>How are you transmuting a &amp;mut [u8] to a &amp;mut Vec&lt;String&gt; ?</p>
</blockquote>
<p>Why, <code>std::mem::transmute</code> of course. ;)</p>
<p>I'm apparently the sort of person who keeps nice people like <span class="user-mention" data-user-id="120791">@RalfJ</span> up at night.</p>



<a name="176044907"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation%20UB/near/176044907" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> RalfJ <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation.20UB.html#176044907">(Sep 18 2019 at 20:47)</a>:</h4>
<blockquote>
<p>Why, std::mem::transmute of course. ;)</p>
</blockquote>
<p>but they don't have the same size.^^</p>



<a name="176044969"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation%20UB/near/176044969" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> RalfJ <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation.20UB.html#176044969">(Sep 18 2019 at 20:48)</a>:</h4>
<blockquote>
<p>I'm apparently the sort of person who keeps nice people like @RalfJ up at night.</p>
</blockquote>
<p>why, in this thread all I did was during "reasonable" times? ;) (I am in UTC+2)</p>



<a name="176045069"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation%20UB/near/176045069" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> RalfJ <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation.20UB.html#176045069">(Sep 18 2019 at 20:49)</a>:</h4>
<blockquote>
<p>At least the language of "points to a valid instance" shakes one up on this (probably healthy). Perhaps we are over-anxious about the potential UB here.</p>
</blockquote>
<p>yeah, well, "valid" is still up in the air. but <code>Vec</code>'s field in question is a <code>Unique&lt;T&gt;</code>, so what is "valid" for <code>Unique&lt;T&gt;</code>? Well, it cannot be <code>dereferencable(size-of-T)</code> because <code>Vec::new()</code> actually returns non-dereferencable stuff...</p>



<a name="176045119"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation%20UB/near/176045119" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> RalfJ <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation.20UB.html#176045119">(Sep 18 2019 at 20:50)</a>:</h4>
<blockquote>
<p>as long as the definition of non-NULL is the same across the encoding and decoding binaries </p>
</blockquote>
<p>oh you people have sick minds.^^<br>
Rust does not support platforms where NULL is not the same as "all bits are 0".</p>



<a name="176074207"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation%20UB/near/176074207" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> gnzlbg <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation.20UB.html#176074207">(Sep 19 2019 at 07:03)</a>:</h4>
<blockquote>
<p>Well, it cannot be dereferencable(size-of-T) because Vec::new() actually returns non-dereferencable stuff...</p>
</blockquote>
<p>There is a "dereferenceable_or_null(size-of-T)" LLVM attribute</p>



<a name="176074280"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation%20UB/near/176074280" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> gnzlbg <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation.20UB.html#176074280">(Sep 19 2019 at 07:04)</a>:</h4>
<p>The problem the pointer of a <code>Vec</code> cannot be dereferenceable is that the size-of-T must be known at compile-time, but for a vec, how many elements you actually have isn't known till run-time</p>



<a name="176074289"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation%20UB/near/176074289" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> gnzlbg <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation.20UB.html#176074289">(Sep 19 2019 at 07:04)</a>:</h4>
<p>There are ways that you could use to communicate this to LLVM though</p>



<a name="176074307"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation%20UB/near/176074307" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> gnzlbg <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation.20UB.html#176074307">(Sep 19 2019 at 07:05)</a>:</h4>
<p>e.g. you could pass the pointer to an identity-like (ptr*, len) -&gt; ptr function that uses the usable_size attribute</p>



<a name="176074311"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation%20UB/near/176074311" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> gnzlbg <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation.20UB.html#176074311">(Sep 19 2019 at 07:05)</a>:</h4>
<p>to tell LLVM that <code>ptr</code> points to an allocation of length <code>len</code></p>



<a name="177088851"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation%20UB/near/177088851" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Hadrien Grasland <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation.20UB.html#177088851">(Oct 01 2019 at 19:23)</a>:</h4>
<p>Err... I... have a new one for this thread.</p>



<a name="177088989"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation%20UB/near/177088989" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Hadrien Grasland <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation.20UB.html#177088989">(Oct 01 2019 at 19:25)</a>:</h4>
<p>Do you think that this evil trick triggers &amp;mut aliasing UB?</p>
<div class="codehilite"><pre><span></span><span class="k">pub</span><span class="w"> </span><span class="k">fn</span> <span class="nf">evil</span><span class="o">&lt;</span><span class="na">&#39;bytes</span><span class="o">&gt;</span><span class="p">(</span><span class="n">bytes</span>: <span class="kp">&amp;</span><span class="na">&#39;bytes</span><span class="w"> </span><span class="k">mut</span><span class="w"> </span><span class="p">[</span><span class="kt">u8</span><span class="p">])</span><span class="w"> </span>-&gt; <span class="kp">&amp;</span><span class="na">&#39;bytes</span><span class="w"> </span><span class="k">mut</span><span class="w"> </span><span class="p">[</span><span class="kt">u8</span><span class="p">]</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w">    </span><span class="k">unsafe</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w">        </span><span class="kd">let</span><span class="w"> </span><span class="n">bytes_uc</span>: <span class="kp">&amp;</span><span class="na">&#39;bytes</span><span class="w"> </span><span class="n">UnsafeCell</span><span class="o">&lt;</span><span class="p">[</span><span class="kt">u8</span><span class="p">]</span><span class="o">&gt;</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">std</span>::<span class="n">mem</span>::<span class="n">transmute</span><span class="p">(</span><span class="n">bytes</span><span class="p">);</span><span class="w"></span>
<span class="w">        </span><span class="p">{</span><span class="w"></span>
<span class="w">            </span><span class="kd">let</span><span class="w"> </span><span class="n">bytes</span>: <span class="kp">&amp;</span><span class="na">&#39;bytes</span><span class="w"> </span><span class="k">mut</span><span class="w"> </span><span class="p">[</span><span class="kt">u8</span><span class="p">]</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="o">&amp;</span><span class="k">mut</span><span class="w"> </span><span class="o">*</span><span class="n">bytes_uc</span><span class="p">.</span><span class="n">get</span><span class="p">();</span><span class="w"></span>
<span class="w">            </span><span class="c1">// ... do stuff with &quot;bytes&quot;, without leaking references out of the scope ...</span>
<span class="w">        </span><span class="p">}</span><span class="w"></span>
<span class="w">        </span><span class="kd">let</span><span class="w"> </span><span class="n">bytes</span>: <span class="kp">&amp;</span><span class="na">&#39;bytes</span><span class="w"> </span><span class="k">mut</span><span class="w"> </span><span class="p">[</span><span class="kt">u8</span><span class="p">]</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="o">&amp;</span><span class="k">mut</span><span class="w"> </span><span class="o">*</span><span class="n">bytes_uc</span><span class="p">.</span><span class="n">get</span><span class="p">();</span><span class="w"></span>
<span class="w">        </span><span class="n">bytes</span><span class="w"></span>
<span class="w">    </span><span class="p">}</span><span class="w"></span>
<span class="p">}</span><span class="w"></span>
</pre></div>



<a name="177089098"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation%20UB/near/177089098" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Hadrien Grasland <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation.20UB.html#177089098">(Oct 01 2019 at 19:26)</a>:</h4>
<p>I think it doesn't, and my rationale is that "bytes" is moved out of the way by the initial transmute, and only "bytes_uc" remains. Therefore, we only get an <code>&amp;'bytes UnsafeCell&lt;[u8]&gt;</code> which aliases with an <code>&amp;'bytes mut [u8]</code>, which is fine. But I might be wrong.</p>



<a name="177090009"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation%20UB/near/177090009" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Hadrien Grasland <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation.20UB.html#177090009">(Oct 01 2019 at 19:36)</a>:</h4>
<p>For context, this is a dirty way to work around <a href="https://users.rust-lang.org/t/is-this-supposed-to-work-with-nll/33090/15" target="_blank" title="https://users.rust-lang.org/t/is-this-supposed-to-work-with-nll/33090/15">an unpleasant situation</a> where 1/the borrow checker decides to borrow "bytes" forever, 2/while I understand where this borrow lifetime comes from, I believe that it is overly pessimistic for my use case, and 3/I do not manage to find an elegant way to rephrase my API so that the borrow checker doesn't do that.</p>



<a name="177095939"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation%20UB/near/177095939" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Hadrien Grasland <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation.20UB.html#177095939">(Oct 01 2019 at 20:40)</a>:</h4>
<p>OK... as it turns out, my actual API does not take an <code>&amp;'bytes mut [u8]</code> but a generic <code>S: DerefMut&lt;Target=[u8]&gt; + 'bytes</code>, and this means that I can use <code>UnsafeCell</code> in a much more straightforward way which I'm pretty sure is correct:</p>
<div class="codehilite"><pre><span></span><span class="k">pub</span><span class="w"> </span><span class="k">fn</span> <span class="nf">less_evil</span><span class="o">&lt;</span><span class="na">&#39;bytes</span><span class="p">,</span><span class="w"> </span><span class="n">S</span>: <span class="nc">DerefMut</span><span class="o">&lt;</span><span class="n">Target</span><span class="o">=</span><span class="p">[</span><span class="kt">u8</span><span class="p">]</span><span class="o">&gt;</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="na">&#39;bytes</span><span class="o">&gt;</span><span class="p">(</span><span class="n">bytes</span>: <span class="nc">S</span><span class="p">)</span><span class="w"> </span>-&gt; <span class="nc">S</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w">    </span><span class="k">unsafe</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w">        </span><span class="kd">let</span><span class="w"> </span><span class="n">bytes_uc</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">UnsafeCell</span>::<span class="n">new</span><span class="p">(</span><span class="n">bytes</span><span class="p">);</span><span class="w"></span>
<span class="w">        </span><span class="p">{</span><span class="w"></span>
<span class="w">            </span><span class="kd">let</span><span class="w"> </span><span class="n">bytes</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="o">&amp;</span><span class="na">&#39;bytes</span><span class="w"> </span><span class="k">mut</span><span class="w"> </span><span class="p">[</span><span class="kt">u8</span><span class="p">]</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">(</span><span class="o">*</span><span class="n">bytes_uc</span><span class="p">.</span><span class="n">get</span><span class="p">()).</span><span class="n">deref_mut</span><span class="p">();</span><span class="w"></span>
<span class="w">            </span><span class="c1">// ... do whatever horrible stuff is necessary without leaking references ...</span>
<span class="w">        </span><span class="p">}</span><span class="w"></span>
<span class="w">        </span><span class="n">bytes_uc</span><span class="p">.</span><span class="n">into_inner</span><span class="p">()</span><span class="w"></span>
<span class="w">    </span><span class="p">}</span><span class="w"></span>
<span class="p">}</span><span class="w"></span>
</pre></div>


<p>So... problem solved, perhaps? Although I'm still interested in 1/API design tips to avoid perpetual borrows (thus removing the need for this unsafe trickery entirely), and 2/opinions on whether the above trick was UB or not. I might end up wanting to use it again someday...</p>



<a name="177099274"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation%20UB/near/177099274" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> HeroicKatora <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation.20UB.html#177099274">(Oct 01 2019 at 21:21)</a>:</h4>
<p>Why are you <em>more</em> sure? <code>DerefMut</code> is a user trait that can call arbitrary code whereas the previous code block had effectively only pointer casts. <code>UnsafeCell</code> only covers <code>S</code> and not the dreferenced bytes. I'm still not sure why you'd need an <code>UnsafeCell</code> in the first place because that doesn't contain any conflicting <em>non-mutable</em> borrows that you somehow dynamically check with synchronization or other abominations. The lifetime ascription <code>let bytes: &amp;'bytes mut [u8]</code> looks wrong now, the type <code>S</code> lives at least that long but that doesn mean <code>bytes</code> does.</p>



<a name="177100977"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation%20UB/near/177100977" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> HeroicKatora <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation.20UB.html#177100977">(Oct 01 2019 at 21:48)</a>:</h4>
<p>The comments also mention <code>Clones</code> and <code>Copies</code> of bytes but it does no such thing, it only creates new references to the very same bytes.<br>
<code>UnsafeCell</code> is probably not the right tool for getting rid of borrow check, simply casting the reference to a pointer achieves the very same thing without that. I also don't get what <code>rustc</code> is supposedly complaining about in the first place. Here is your example without any <code>UnsafeCell</code> or any other <code>unsafe</code> in <code>evil_foo</code> in fact. This appears to have the exact same observable effects, in my eyes, and compiles just fine.<br>
<a href="https://play.rust-lang.org/?version=stable&amp;mode=debug&amp;edition=2018&amp;gist=49e0fc46352010e04f6e811c4e4f6c77" target="_blank" title="https://play.rust-lang.org/?version=stable&amp;mode=debug&amp;edition=2018&amp;gist=49e0fc46352010e04f6e811c4e4f6c77">https://play.rust-lang.org/?version=stable&amp;mode=debug&amp;edition=2018&amp;gist=49e0fc46352010e04f6e811c4e4f6c77</a><br>
It may be an issue that your type ascriptions are <em>too strong</em>. For example, manually mentioning lifetime <code>'bytes</code> <em>forces</em> the compiler to borrow the object for that whole lifetime, which actively sabotages any possible shorter lifetime that NLL may be able to assign to that reference.</p>



<a name="177712889"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation%20UB/near/177712889" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> RalfJ <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation.20UB.html#177712889">(Oct 09 2019 at 13:24)</a>:</h4>
<blockquote>
<p>Do you think that this evil trick triggers &amp;mut aliasing UB?</p>
</blockquote>
<p>for Stacked Borrows, lifetimes don't matter. so the innermost <code>bytes</code> having a longer lifetime does not matter, so I think under Stacked Borrows your first snippet is okay. did you try running it in Miri with a few input sizes?<br>
but it does make me wonder why you need that "fake" long lifetime? certainly if any reference derived from that is used after the outer <code>bytes</code> gets created, you got UB. (not using the same variable name 3 times would make this code easier to discuss...)</p>



<a name="177712894"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation%20UB/near/177712894" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> RalfJ <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation.20UB.html#177712894">(Oct 09 2019 at 13:24)</a>:</h4>
<p><span class="user-mention" data-user-id="211871">@Hadrien Grasland</span> ^</p>



<a name="177732723"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation%20UB/near/177732723" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Hadrien Grasland <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation.20UB.html#177732723">(Oct 09 2019 at 16:46)</a>:</h4>
<p>Whoops, for some reason zulip didn't send me a notification e-mail when I expected it to. Apologies, <span class="user-mention" data-user-id="229913">@HeroicKatora</span> .</p>



<a name="177732860"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation%20UB/near/177732860" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Hadrien Grasland <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation.20UB.html#177732860">(Oct 09 2019 at 16:48)</a>:</h4>
<p>Anyhow, here's more context on what I'm trying to do.</p>



<a name="177733092"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation%20UB/near/177733092" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Hadrien Grasland <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation.20UB.html#177733092">(Oct 09 2019 at 16:51)</a>:</h4>
<p>Currently, abomonation can only serialize types which own all of their data, such as <code>bool</code> or <code>Box&lt;T&gt;</code> where <code>T: 'static</code>. This limitation is artificial though: if we can serialize heap-allocated data, we can serialize stack references in exactly the same manner. As I recently ended up in a situation where being able to serialize a type which contains references would be convenient, I tried to implement abomonation support for those.</p>



<a name="177733242"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation%20UB/near/177733242" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Hadrien Grasland <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation.20UB.html#177733242">(Oct 09 2019 at 16:52)</a>:</h4>
<p>...but then I found out an unpleasant truth which had been known in serde circles for a while: although _serializing_ data which contains references is fine, _deserializing_ such data must be done with care.</p>



<a name="177733557"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation%20UB/near/177733557" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Hadrien Grasland <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation.20UB.html#177733557">(Oct 09 2019 at 16:56)</a>:</h4>
<p>To see why, consider the following type, for example:</p>
<div class="codehilite"><pre><span></span><span class="k">struct</span> <span class="nc">Foo</span><span class="o">&lt;</span><span class="na">&#39;y</span><span class="o">&gt;</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w">    </span><span class="n">x</span>: <span class="kt">bool</span><span class="p">,</span><span class="w"></span>
<span class="w">    </span><span class="n">y</span>: <span class="kp">&amp;</span><span class="na">&#39;y</span><span class="w"> </span><span class="kt">str</span><span class="p">,</span><span class="w"></span>
<span class="p">}</span><span class="w"></span>
</pre></div>


<p>Serializing a <code>Foo&lt;'y&gt;</code> of any lifetime is fine. abomonation will just memcpy the <code>Foo&lt;'y&gt;</code>'s data into the target bytes, followed by a memcpy of the data of string <code>y</code>. So far, so good.</p>



<a name="177733940"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation%20UB/near/177733940" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Hadrien Grasland <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation.20UB.html#177733940">(Oct 09 2019 at 17:00)</a>:</h4>
<p>However, in-place deserialization of <code>Foo&lt;'y&gt;</code> from <code>&amp;'bytes mut [u8]</code> is only fine if <code>'bytes: 'y</code>, because during the deserialization process, abomonation will overwrite <code>Foo</code>'s <code>y</code> field to point to neighboring bytes in the serialized data. This means that abomonation will actually put an <code>&amp;'bytes str</code> where an <code>&amp;'y str</code> is expected. Which is only memory-safe (and therefore respectful of Rust's reference subtyping rules) when <code>&amp;'bytes str</code> is strictly longer-lived than <code>&amp;'y str</code>, i.e. <code>'bytes: 'y</code>.</p>



<a name="177734164"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation%20UB/near/177734164" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Hadrien Grasland <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation.20UB.html#177734164">(Oct 09 2019 at 17:02)</a>:</h4>
<p>Which means that, for example, abomonation should not allow deserialization of a <code>Foo&lt;'static&gt;</code> from stack-allocated bytes, because then the user could make a copy of _what rustc thinks of as a <code>&amp;'static str</code>_, drop the bytes, and get a dangling reference.</p>



<a name="177734396"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation%20UB/near/177734396" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Hadrien Grasland <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation.20UB.html#177734396">(Oct 09 2019 at 17:04)</a>:</h4>
<p>Now, as mentioned before, this problem was already encountered by serde, and a fine solution was found there: bound the deserialization trait and function so that only memory-safe deserializations are allowed. This is the meaning of the <code>'de</code> lifetime in serde's <code>Deserialize&lt;'de&gt;</code> trait, <a href="https://serde.rs/lifetimes.html" target="_blank" title="https://serde.rs/lifetimes.html">as explained here</a>.</p>



<a name="177734934"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation%20UB/near/177734934" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Hadrien Grasland <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation.20UB.html#177734934">(Oct 09 2019 at 17:09)</a>:</h4>
<p>That design generally made sense in abomonation too, so I mostly reused it as-is, ending up with this deserialization function signature:</p>
<div class="codehilite"><pre><span></span><span class="k">pub</span><span class="w"> </span><span class="k">unsafe</span><span class="w"> </span><span class="k">fn</span> <span class="nf">decode</span><span class="o">&lt;</span><span class="na">&#39;bytes</span><span class="p">,</span><span class="w"> </span><span class="n">T</span>: <span class="nc">Exhume</span><span class="o">&lt;</span><span class="na">&#39;bytes</span><span class="o">&gt;&gt;</span><span class="p">(</span><span class="n">bytes</span>: <span class="kp">&amp;</span><span class="na">&#39;bytes</span><span class="w"> </span><span class="k">mut</span><span class="w"> </span><span class="p">[</span><span class="kt">u8</span><span class="p">])</span><span class="w"> </span>-&gt; <span class="nb">Option</span><span class="o">&lt;</span><span class="p">(</span><span class="o">&amp;</span><span class="na">&#39;bytes</span><span class="w"> </span><span class="n">T</span><span class="p">,</span><span class="w"> </span><span class="o">&amp;</span><span class="na">&#39;bytes</span><span class="w"> </span><span class="k">mut</span><span class="w"> </span><span class="p">[</span><span class="kt">u8</span><span class="p">])</span><span class="o">&gt;</span><span class="w"></span>
</pre></div>


<p>But then I encountered trouble while porting an abomonation feature which serde doesn't have, namely the <code>Abomonated</code> type.</p>



<a name="177735570"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation%20UB/near/177735570" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Hadrien Grasland <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation.20UB.html#177735570">(Oct 09 2019 at 17:16)</a>:</h4>
<p>You see, like any reference-based abstraction, abomonation deserialization has this problem that it's hard to move your deserialized data around in the program. One could try to use something like owning-ref in order to bring the serialized data and corresponding bytes together, but then one ends up with a self-referential type, which is Rust jargon for intense self-imposed pain.</p>



<a name="177735957"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation%20UB/near/177735957" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Hadrien Grasland <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation.20UB.html#177735957">(Oct 09 2019 at 17:20)</a>:</h4>
<p>As an alternative, abomonation provides built-in support for owned deserialized data in the form of the <code>Abomonated</code> type, which is essentially an owned slice of bytes that happens to contain a correct binary representation for deserialized data of a certain type T, and implements a <code>Deref&lt;Target=T&gt;</code> that simply casts a pointer to the beginning to that slice of bytes into an <code>&amp;T</code>.</p>



<a name="177736377"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation%20UB/near/177736377" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Hadrien Grasland <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation.20UB.html#177736377">(Oct 09 2019 at 17:25)</a>:</h4>
<p>(Now, the astute reader will have noticed that this actually cannot work with all "slice of bytes" types, because moving e.g. an [u8; N] will result in inner self-pointer invalidation. Well, that's why <code>Abomonated</code>'s constructor is unsafe: we accept any <code>DerefMut&lt;Target=T&gt;</code>, but we actually would like to accept something more restrictive like owning-ref's <code>StableDeref</code>. Since Rust's stdlib has nothing of the sort at this point in time, we make the constructor unsafe and ask users to please not build an Abomonated out of those owned slices of bytes. Anyway, that's not the heart of the issue here...)</p>



<a name="177737664"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation%20UB/near/177737664" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Hadrien Grasland <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation.20UB.html#177737664">(Oct 09 2019 at 17:38)</a>:</h4>
<p>Now, before abomonation had reference support, <code>Abomonated</code> was defined like this...</p>
<div class="codehilite"><pre><span></span><span class="k">pub</span><span class="w"> </span><span class="k">struct</span> <span class="nc">Abomonated</span><span class="o">&lt;</span><span class="n">T</span><span class="p">,</span><span class="w"> </span><span class="n">S</span>: <span class="nc">DerefMut</span><span class="o">&lt;</span><span class="n">Target</span><span class="o">=</span><span class="p">[</span><span class="kt">u8</span><span class="p">]</span><span class="o">&gt;&gt;</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w">    </span><span class="n">phantom</span>: <span class="nc">PhantomData</span><span class="o">&lt;</span><span class="n">T</span><span class="o">&gt;</span><span class="p">,</span><span class="w"></span>
<span class="w">    </span><span class="n">decoded</span>: <span class="nc">S</span><span class="p">,</span><span class="w"></span>
<span class="p">}</span><span class="w"></span>
</pre></div>


<p>...and its constructor was defined in an <code>impl</code> block that looked like this:</p>
<div class="codehilite"><pre><span></span><span class="k">impl</span><span class="o">&lt;</span><span class="n">T</span>: <span class="nc">Abomonation</span><span class="p">,</span><span class="w"> </span><span class="n">S</span>: <span class="nc">DerefMut</span><span class="o">&lt;</span><span class="n">Target</span><span class="o">=</span><span class="p">[</span><span class="kt">u8</span><span class="p">]</span><span class="o">&gt;&gt;</span><span class="w"> </span><span class="n">Abomonated</span><span class="o">&lt;</span><span class="n">T</span><span class="p">,</span><span class="w"> </span><span class="n">S</span><span class="o">&gt;</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w">    </span><span class="c1">// ... blah blah blah thid code will eat your cat if S is an array blah blah blah</span>
<span class="w">    </span><span class="k">pub</span><span class="w"> </span><span class="k">unsafe</span><span class="w"> </span><span class="k">fn</span> <span class="nf">new</span><span class="p">(</span><span class="k">mut</span><span class="w"> </span><span class="n">bytes</span>: <span class="nc">S</span><span class="p">)</span><span class="w"> </span>-&gt; <span class="nb">Option</span><span class="o">&lt;</span><span class="n">Self</span><span class="o">&gt;</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w">        </span><span class="c1">// performs the underlying pointer correction, returns None on failure.</span>
<span class="w">        </span><span class="kd">let</span><span class="w"> </span><span class="n">_decoded</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">decode</span>::<span class="o">&lt;</span><span class="n">T</span><span class="o">&gt;</span><span class="p">(</span><span class="n">bytes</span><span class="p">.</span><span class="n">deref_mut</span><span class="p">())</span><span class="o">?</span><span class="p">;</span><span class="w"></span>
<span class="w">        </span><span class="nb">Some</span><span class="p">(</span><span class="n">Abomonated</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w">            </span><span class="n">phantom</span>: <span class="nc">PhantomData</span><span class="p">,</span><span class="w"></span>
<span class="w">            </span><span class="n">decoded</span>: <span class="nc">bytes</span><span class="p">,</span><span class="w"></span>
<span class="w">        </span><span class="p">})</span><span class="w"></span>
<span class="w">    </span><span class="p">}</span><span class="w"></span>
<span class="p">}</span><span class="w"></span>
</pre></div>


<p>...but now I have this <code>'bytes</code> lifetime to take care of. Essentially, I must check that S lives long enough to make deserialization of a T from it sensible. This is what I ended up with:</p>
<div class="codehilite"><pre><span></span><span class="k">impl</span><span class="o">&lt;</span><span class="na">&#39;bytes</span><span class="p">,</span><span class="w"> </span><span class="n">T</span><span class="p">,</span><span class="w"> </span><span class="n">S</span><span class="o">&gt;</span><span class="w"> </span><span class="n">Abomonated</span><span class="o">&lt;</span><span class="n">T</span><span class="p">,</span><span class="w"> </span><span class="n">S</span><span class="o">&gt;</span><span class="w"></span>
<span class="w">    </span><span class="k">where</span><span class="w"> </span><span class="n">S</span>: <span class="nc">DerefMut</span><span class="o">&lt;</span><span class="n">Target</span><span class="o">=</span><span class="p">[</span><span class="kt">u8</span><span class="p">]</span><span class="o">&gt;</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="na">&#39;bytes</span><span class="p">,</span><span class="w"></span>
<span class="w">          </span><span class="n">T</span>: <span class="nc">Abomonation</span><span class="o">&lt;</span><span class="na">&#39;bytes</span><span class="o">&gt;</span><span class="p">,</span><span class="w"></span>
<span class="p">{</span><span class="w"></span>
</pre></div>


<p>...where <code>Abomonation&lt;'bytes&gt;</code> is a supertrait of <code>Exhume&lt;'bytes&gt;</code> which also includes serialization (which TBH I currently don't need for <code>Abomonated</code>, <code>Exhume&lt;'bytes&gt;</code> would also work).</p>



<a name="177737844"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation%20UB/near/177737844" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Hadrien Grasland <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation.20UB.html#177737844">(Oct 09 2019 at 17:40)</a>:</h4>
<p>However, doing that effectively borrows <code>bytes</code> forever as soon as <code>decode()</code> is called in <code>Abomonated::new()</code>, which is how I ended up engaging in the kind of horrible borrow checker workarounds described earlier.</p>



<a name="177738059"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation%20UB/near/177738059" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Hadrien Grasland <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation.20UB.html#177738059">(Oct 09 2019 at 17:42)</a>:</h4>
<p>I tried various variants, including variations of...</p>
<div class="codehilite"><pre><span></span><span class="k">impl</span><span class="o">&lt;</span><span class="na">&#39;bytes</span><span class="p">,</span><span class="w"> </span><span class="na">&#39;abo</span><span class="p">,</span><span class="w"> </span><span class="n">S</span>: <span class="nc">DerefMut</span><span class="o">&lt;</span><span class="n">Target</span><span class="o">=</span><span class="p">[</span><span class="kt">u8</span><span class="p">]</span><span class="o">&gt;</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="na">&#39;bytes</span><span class="p">,</span><span class="w"> </span><span class="n">T</span>: <span class="nc">Abomonation</span><span class="o">&lt;</span><span class="na">&#39;abo</span><span class="o">&gt;&gt;</span><span class="w"> </span><span class="n">Abomonated</span><span class="o">&lt;</span><span class="n">T</span><span class="p">,</span><span class="w"> </span><span class="n">S</span><span class="o">&gt;</span><span class="w"></span>
</pre></div>


<p>...but none could avoid the "slice borrowed forever" problem. And in the end, I convinced myself that this was an intrinsic defect of the kind of API that I had built for <code>decode()</code> above. For which I don't have a better idea...</p>



<a name="177738279"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation%20UB/near/177738279" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Hadrien Grasland <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation.20UB.html#177738279">(Oct 09 2019 at 17:45)</a>:</h4>
<p>Oh, well, now I'm taking a shortcut. Actually, there is one signature that almost got there, which used HRTBs...</p>
<div class="codehilite"><pre><span></span><span class="k">impl</span><span class="o">&lt;</span><span class="n">T</span><span class="p">,</span><span class="w"> </span><span class="n">S</span>: <span class="nc">DerefMut</span><span class="o">&lt;</span><span class="n">Target</span><span class="o">=</span><span class="p">[</span><span class="kt">u8</span><span class="p">]</span><span class="o">&gt;&gt;</span><span class="w"> </span><span class="n">Abomonated</span><span class="o">&lt;</span><span class="n">T</span><span class="p">,</span><span class="w"> </span><span class="n">S</span><span class="o">&gt;</span><span class="w"></span>
<span class="w">    </span><span class="k">where</span><span class="w"> </span><span class="k">for</span><span class="o">&lt;</span><span class="na">&#39;abo</span><span class="o">&gt;</span><span class="w"> </span><span class="n">T</span>: <span class="nc">Abomonation</span><span class="o">&lt;</span><span class="na">&#39;abo</span><span class="o">&gt;</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
</pre></div>


<p>...but after a while, I realized that this excluded use of any type <code>T</code> that contains references with <code>Abomonated</code> which isn't terribly satisfying either.</p>



<a name="177738872"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation%20UB/near/177738872" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Hadrien Grasland <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation.20UB.html#177738872">(Oct 09 2019 at 17:51)</a>:</h4>
<p>In the end, what I am doing _is_ kind of fishy. After all, my bytes do end up containing self-referential deserialized data, even if <code>rustc</code> doesn't really know about that fact. Therefore, I'm not entirely surprised that building <code>Abomonated</code> entails working around the borrow checker to some degree.</p>



<a name="177738952"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation%20UB/near/177738952" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Hadrien Grasland <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation.20UB.html#177738952">(Oct 09 2019 at 17:52)</a>:</h4>
<p>OTOH, I'd like my workaround to be correct, and you seem to imply that it isn't yet. Need to think about it some more, then...</p>



<a name="177740567"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation%20UB/near/177740567" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Hadrien Grasland <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation.20UB.html#177740567">(Oct 09 2019 at 18:09)</a>:</h4>
<p>More generally, though, this <code>decode()</code> API redesign has been a source of pain in other areas, such as unit tests. Previously, the part of abomonation's unit tests that makes sure round trip serialization works used to look like this:</p>
<div class="codehilite"><pre><span></span><span class="k">fn</span> <span class="nf">_test_pass</span><span class="o">&lt;</span><span class="n">T</span>: <span class="nc">Abomonation</span><span class="o">+</span><span class="nb">Eq</span><span class="o">&gt;</span><span class="p">(</span><span class="n">record</span>: <span class="nc">T</span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w">    </span><span class="kd">let</span><span class="w"> </span><span class="k">mut</span><span class="w"> </span><span class="n">bytes</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nb">Vec</span>::<span class="n">new</span><span class="p">();</span><span class="w"></span>
<span class="w">    </span><span class="k">unsafe</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="n">encode</span><span class="p">(</span><span class="o">&amp;</span><span class="n">record</span><span class="p">,</span><span class="w"> </span><span class="o">&amp;</span><span class="k">mut</span><span class="w"> </span><span class="n">bytes</span><span class="p">).</span><span class="n">unwrap</span><span class="p">();</span><span class="w"> </span><span class="p">}</span><span class="w"></span>
<span class="w">    </span><span class="p">{</span><span class="w"></span>
<span class="w">        </span><span class="kd">let</span><span class="w"> </span><span class="p">(</span><span class="n">result</span><span class="p">,</span><span class="w"> </span><span class="n">rest</span><span class="p">)</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="k">unsafe</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="n">decode</span>::<span class="o">&lt;</span><span class="n">T</span><span class="o">&gt;</span><span class="p">(</span><span class="o">&amp;</span><span class="k">mut</span><span class="w"> </span><span class="n">bytes</span><span class="p">[..])</span><span class="w"> </span><span class="p">}.</span><span class="n">unwrap</span><span class="p">();</span><span class="w"></span>
<span class="w">        </span><span class="n">assert</span><span class="o">!</span><span class="p">(</span><span class="o">&amp;</span><span class="n">record</span><span class="w"> </span><span class="o">==</span><span class="w"> </span><span class="n">result</span><span class="p">);</span><span class="w"></span>
<span class="w">        </span><span class="n">assert</span><span class="o">!</span><span class="p">(</span><span class="n">rest</span><span class="p">.</span><span class="n">len</span><span class="p">()</span><span class="w"> </span><span class="o">==</span><span class="w"> </span><span class="mi">0</span><span class="p">);</span><span class="w"></span>
<span class="w">    </span><span class="p">}</span><span class="w"></span>
<span class="p">}</span><span class="w"></span>
</pre></div>


<p>Take abomonable data, serialize it, deserialize it, and make sure that we get the same data. Simple enough.</p>
<p>However, in <a href="https://github.com/TimelyDataflow/abomonation/pull/28" target="_blank" title="https://github.com/TimelyDataflow/abomonation/pull/28">my PR for reference support</a>, I had to change that test into this:</p>
<div class="codehilite"><pre><span></span><span class="c1">// FIXME: I could not find an API which allows _test_pass to allocate a Vec</span>
<span class="c1">//        internally without restricting the set of allowed Abomonation impls.</span>
<span class="k">fn</span> <span class="nf">_test_pass</span><span class="o">&lt;</span><span class="na">&#39;bytes</span><span class="p">,</span><span class="w"> </span><span class="n">T</span><span class="o">&gt;</span><span class="p">(</span><span class="k">mut</span><span class="w"> </span><span class="n">bytes</span>: <span class="kp">&amp;</span><span class="na">&#39;bytes</span><span class="w"> </span><span class="k">mut</span><span class="w"> </span><span class="nb">Vec</span><span class="o">&lt;</span><span class="kt">u8</span><span class="o">&gt;</span><span class="p">,</span><span class="w"> </span><span class="n">record</span>: <span class="nc">T</span><span class="p">)</span><span class="w"></span>
<span class="w">    </span><span class="k">where</span><span class="w"> </span><span class="n">T</span>: <span class="nc">Abomonation</span><span class="o">&lt;</span><span class="na">&#39;bytes</span><span class="o">&gt;</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="n">Debug</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="nb">Eq</span><span class="w"></span>
<span class="p">{</span><span class="w"></span>
<span class="w">    </span><span class="k">unsafe</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="n">encode</span><span class="p">(</span><span class="o">&amp;</span><span class="n">record</span><span class="p">,</span><span class="w"> </span><span class="o">&amp;</span><span class="k">mut</span><span class="w"> </span><span class="n">bytes</span><span class="p">).</span><span class="n">unwrap</span><span class="p">();</span><span class="w"> </span><span class="p">}</span><span class="w"></span>
<span class="w">    </span><span class="p">{</span><span class="w"></span>
<span class="w">        </span><span class="kd">let</span><span class="w"> </span><span class="p">(</span><span class="n">result</span><span class="p">,</span><span class="w"> </span><span class="n">rest</span><span class="p">)</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="k">unsafe</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="n">decode</span>::<span class="o">&lt;</span><span class="n">T</span><span class="o">&gt;</span><span class="p">(</span><span class="o">&amp;</span><span class="k">mut</span><span class="w"> </span><span class="n">bytes</span><span class="p">[..])</span><span class="w"> </span><span class="p">}.</span><span class="n">unwrap</span><span class="p">();</span><span class="w"></span>
<span class="w">        </span><span class="n">assert_eq</span><span class="o">!</span><span class="p">(</span><span class="o">&amp;</span><span class="n">record</span><span class="p">,</span><span class="w"> </span><span class="n">result</span><span class="p">);</span><span class="w"></span>
<span class="w">        </span><span class="n">assert_eq</span><span class="o">!</span><span class="p">(</span><span class="n">rest</span><span class="p">.</span><span class="n">len</span><span class="p">(),</span><span class="w"> </span><span class="mi">0</span><span class="p">);</span><span class="w"></span>
<span class="w">    </span><span class="p">}</span><span class="w"></span>
<span class="p">}</span><span class="w"></span>
</pre></div>


<p>Notice that the test must now take a <code>Vec</code> as input, instead of being allowed to create said <code>Vec</code> internally. This is the only trick I've found to express an <code>Abomonation</code> trait bound that is compatible with reference-based types so far. Again, higher-ranked trait bounds would forbid testing of any type that contains references altogether...</p>



<a name="177740864"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation%20UB/near/177740864" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Hadrien Grasland <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation.20UB.html#177740864">(Oct 09 2019 at 18:12)</a>:</h4>
<p>Now, this specific change feels just wrong. There has to be a way to express the deserialization lifetime constraint of types that contain references, in a fashion that does not randomly break clients of <code>decode()</code>. But although I've been trying various things for a while, I have yet to find that way.</p>



<a name="177742421"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation%20UB/near/177742421" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Hadrien Grasland <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation.20UB.html#177742421">(Oct 09 2019 at 18:29)</a>:</h4>
<p>Hope that clarifies the situation. I've tried hard to provide reduced toy examples that avoid spelling out the full complexity of the problem, but it seems that people always end up wondering what's the problem when looking only at the reduced examples ;)</p>



<a name="177781650"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation%20UB/near/177781650" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> comex <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation.20UB.html#177781650">(Oct 10 2019 at 05:57)</a>:</h4>
<p>Have you tried it in miri?</p>



<a name="177782525"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation%20UB/near/177782525" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Hadrien Grasland <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation.20UB.html#177782525">(Oct 10 2019 at 06:19)</a>:</h4>
<p>Miri's happy with the UnsafeCell-based version, as well as a different version that just creates two <code>&amp;mut [u8]</code> to the same bytes without <code>mem::forget</code>-ing the first one before using the second one. I thought that was UB, but apparently it's not under Stacked Borrows / NLL.</p>



<a name="177788940"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation%20UB/near/177788940" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> RalfJ <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation.20UB.html#177788940">(Oct 10 2019 at 08:22)</a>:</h4>
<blockquote>
<p>Miri's happy with the UnsafeCell-based version, as well as a different version that just creates two <code>&amp;mut [u8]</code> to the same bytes without <code>mem::forget</code>-ing the first one before using the second one. I thought that was UB, but apparently it's not under Stacked Borrows / NLL.</p>
</blockquote>
<p>it is UB if the liveness ranges of the two references overlap</p>



<a name="177788958"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation%20UB/near/177788958" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> RalfJ <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation.20UB.html#177788958">(Oct 10 2019 at 08:23)</a>:</h4>
<p>and as explained elsewhere, <code>mem::forget</code> on a reference is a bad idea. all it does is extend its liveness range because you are adding another "use"</p>



<a name="177789015"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation%20UB/near/177789015" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> RalfJ <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation.20UB.html#177789015">(Oct 10 2019 at 08:24)</a>:</h4>
<p><span class="user-mention" data-user-id="211871">@Hadrien Grasland</span> you seem to be thinking along the lines of my 2017 "types as contracts" proposal, but that turned out to be unworkable. too complicated, and too much existing code it is not compatible with.</p>



<a name="177795508"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation%20UB/near/177795508" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Hadrien Grasland <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation.20UB.html#177795508">(Oct 10 2019 at 10:06)</a>:</h4>
<blockquote>
<blockquote>
<p>Miri's happy with the UnsafeCell-based version, as well as a different version that just creates two <code>&amp;mut [u8]</code> to the same bytes without <code>mem::forget</code>-ing the first one before using the second one. I thought that was UB, but apparently it's not under Stacked Borrows / NLL.</p>
</blockquote>
<p>it is UB if the liveness ranges of the two references overlap</p>
</blockquote>
<p>Yes, and I am uneasy about that. I would really like to make it a compiler error to reuse the first reference after creating the second one, as opposed to silent UB that is only detected when running miri. If allowed, <code>mem::forget</code> would have allowed me to clarify that problem in the code.</p>



<a name="177839724"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation%20UB/near/177839724" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> RalfJ <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation.20UB.html#177839724">(Oct 10 2019 at 18:33)</a>:</h4>
<p>you could try moving instead of reborrowing by adding some braces</p>



<a name="177839734"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation%20UB/near/177839734" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> RalfJ <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation.20UB.html#177839734">(Oct 10 2019 at 18:33)</a>:</h4>
<p><code>{bytes}</code> instead of <code>bytes</code></p>



<a name="177840507"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation%20UB/near/177840507" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Hadrien Grasland <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation.20UB.html#177840507">(Oct 10 2019 at 18:39)</a>:</h4>
<p>May I ask for a slightly more fleshed out code example, starting from this base?</p>
<div class="codehilite"><pre><span></span><span class="k">pub</span><span class="w"> </span><span class="k">fn</span> <span class="nf">evil_foo</span><span class="o">&lt;</span><span class="na">&#39;bytes</span><span class="p">,</span><span class="w"> </span><span class="n">T</span><span class="o">&gt;</span><span class="p">(</span><span class="n">bytes</span>: <span class="kp">&amp;</span><span class="na">&#39;bytes</span><span class="w"> </span><span class="k">mut</span><span class="w"> </span><span class="p">[</span><span class="kt">u8</span><span class="p">])</span><span class="w"> </span>-&gt; <span class="kp">&amp;</span><span class="na">&#39;bytes</span><span class="w"> </span><span class="k">mut</span><span class="w"> </span><span class="p">[</span><span class="kt">u8</span><span class="p">]</span><span class="w"></span>
<span class="w">    </span><span class="k">where</span><span class="w"> </span><span class="n">T</span>: <span class="nc">InPlaceDeserialize</span><span class="o">&lt;</span><span class="na">&#39;bytes</span><span class="o">&gt;</span><span class="w"></span>
<span class="p">{</span><span class="w"></span>
<span class="w">    </span><span class="k">unsafe</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w">        </span><span class="kd">let</span><span class="w"> </span><span class="p">(</span><span class="n">ptr</span><span class="p">,</span><span class="w"> </span><span class="n">len</span><span class="p">)</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">(</span><span class="n">bytes</span><span class="p">.</span><span class="n">as_mut_ptr</span><span class="p">(),</span><span class="w"> </span><span class="n">bytes</span><span class="p">.</span><span class="n">len</span><span class="p">());</span><span class="w"></span>
<span class="w">        </span><span class="c1">// NOTE: I would like to make the original &quot;bytes&quot; unusable from here on</span>
<span class="w">        </span><span class="c1">//       to avoid accidental &amp;mut aliasing with &quot;bytes2&quot;</span>

<span class="w">        </span><span class="p">{</span><span class="w"></span>
<span class="w">            </span><span class="kd">let</span><span class="w"> </span><span class="n">bytes2</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">std</span>::<span class="n">slice</span>::<span class="n">from_raw_parts_mut</span>::<span class="o">&lt;</span><span class="na">&#39;bytes</span><span class="p">,</span><span class="w"> </span><span class="n">_</span><span class="o">&gt;</span><span class="p">(</span><span class="n">ptr</span><span class="p">,</span><span class="w"> </span><span class="n">len</span><span class="p">);</span><span class="w"></span>
<span class="w">            </span><span class="n">deserialize2</span>::<span class="o">&lt;</span><span class="n">T</span><span class="o">&gt;</span><span class="p">(</span><span class="n">bytes2</span><span class="p">);</span><span class="w"></span>
<span class="w">        </span><span class="p">}</span><span class="w"></span>

<span class="w">        </span><span class="n">std</span>::<span class="n">slice</span>::<span class="n">from_raw_parts_mut</span>::<span class="o">&lt;</span><span class="na">&#39;bytes</span><span class="p">,</span><span class="w"> </span><span class="n">_</span><span class="o">&gt;</span><span class="p">(</span><span class="n">ptr</span><span class="p">,</span><span class="w"> </span><span class="n">len</span><span class="p">)</span><span class="w"></span>
<span class="w">    </span><span class="p">}</span><span class="w"></span>
<span class="p">}</span><span class="w"></span>
</pre></div>



<a name="177840738"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation%20UB/near/177840738" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> RalfJ <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation.20UB.html#177840738">(Oct 10 2019 at 18:41)</a>:</h4>
<p>oh hm there's two uses...<br>
well here's a "silly" solution: add <code>let bytes = ();</code> :P</p>



<a name="177840908"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation%20UB/near/177840908" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> RalfJ <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation.20UB.html#177840908">(Oct 10 2019 at 18:43)</a>:</h4>
<p>shadowing FTW...</p>



<a name="177841371"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation%20UB/near/177841371" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Hadrien Grasland <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation.20UB.html#177841371">(Oct 10 2019 at 18:48)</a>:</h4>
<p>Nice trick, I could live with that!<br>
(IIRC, that's also the trick used by that crate for stack pinning whose name I have forgotten, right?)</p>



<a name="177841629"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation%20UB/near/177841629" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> RalfJ <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation.20UB.html#177841629">(Oct 10 2019 at 18:50)</a>:</h4>
<p>yeah, I've seen it before</p>



<a name="177841892"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation%20UB/near/177841892" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation.20UB.html#177841892">(Oct 10 2019 at 18:53)</a>:</h4>
<p>you can also just do <code>let bytes: ();</code> maybe even without the <code>()</code> annotation -- that'll prevent all uses</p>



<a name="177841918"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation%20UB/near/177841918" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Abomonation.20UB.html#177841918">(Oct 10 2019 at 18:53)</a>:</h4>
<p>though in this case it probably doesn't matter</p>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>